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Letter from the Editor 


This volume of Professional Files brings us two articles on very different topics, but with a | 
common theme—relationships. l 


Wang introduces Social Network Analysis, a data visualization technique that focuses on the Association for 

: . . . . : . . Institutional Research 
relationships among cases, instead of just their attributes. Her three examples applying this 
technique to common IR study questions will make you want to start building your own SNA 


models right away! 


Carpenter-Hubin, Sullivan, and Herbers share their experience building relationships with faculty through a collaborative 
study of faculty workload and resources. Their insights into how to work as peers respectful of each other's expertise can 
serve as a model for our own research partnerships. 


Consider this a reminder to nurture your own relationships with IR colleagues by sharing your work in AIR Professional 
Files! 


Sincerely, 
Dini x Ape ra 


Sharron Ronco 
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Abstract 

Institutional researchers come to their 
field from a variety of educational and 
work experiences. Regardless of their 
expertise, however, it is difficult—if not 
impossible—for a single researcher, 

or even a large team of researchers, to 
know and understand all the nuanced 
differences among the many disciplines 
found in a comprehensive university. 
This paper discusses the collaboration 
between institutional researchers 

and faculty to evaluate the faculty 
experience in science and engineering. 
Rather than discussing the outcome of 
that evaluation, this paper focuses on 
the value of the collaborative process. 


INTRODUCTION 


The Ohio State University’s project 
Comprehensive Equity at Ohio State 


(Project CEOS), which was funded 

by a grant from the National Science 
Foundation, focuses on retention of 
women faculty in the STEM disciplines 
(science, technology, engineering, 

and mathematics). Project CEOS has 
worked intensively with administrators 
and faculty in three STEM colleges 
(Engineering, Veterinary Medicine, 

and the Division of Natural and 
Mathematical Sciences within Arts and 
Sciences). Project CEOS researchers 
wished to study and evaluate resource 
allocation and working environments 
for men and women faculty in these 
units to understand whether Ohio State 
has problems similar to those identified 
in the landmark Massachusetts Institute 
of Technology (MIT) study, “A Study 

on the Status of Women Faculty in 
Science at MIT” (Chisholm et al., 1999), 
described below. 


At the request of Project CEOS, the 

Office of Academic Affairs appointed a 
committee to study resource allocation 
and working environments. The 
committee was charged to identify 
important resource parameters of the 
work environment for faculty, and then to 
measure those parameters appropriately 
in order to ascertain whether gender was 
an explanatory variable. 


The committee included four women 
and five men holding the following 
staff and faculty positions: 


- Assistant vice president, 
Institutional Research and Planning 

+ Associate director, Institutional 
Research and Planning 

- Director, Human Resources 
Organizational Metrics and Data 
Analytics 

+ Professor, Evolution, Ecology, and 
Organismal Biology; principal 
investigator, Project CEOS 

+ Chair, Faculty Compensation and 
Benefits Committee; professor, 
Comparative Studies 

+ Professor, Veterinary Biosciences 

« Associate provost and director, The 
Women's Place; professor, City and 
Regional Planning 

+ Professor, Statistics 

+ Professor, Electrical and Computer 
Engineering 


The results of the committee’s work 
have been published in a special report 
(Herbers & Desai, 2012). The actual 
findings are of interest, but here we 
focus on the value of the collaborative 
process, with particular emphasis 

on the insights brought to bear by 

the faculty who work in the relevant 
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environments and how those insights 
shaped the analysis. Our experiences 
show how understanding patterns 

in data concerning faculty work is 
best achieved through collaboration 
between institutional researchers and 
faculty from multiple disciplines in the 
discussions and analysis. Furthermore, 
such efforts provide faculty with 
opportunities to learn about disciplines 
outside their own. 


COMMITTEE 
BACKGROUND 


“Institutional researchers should seek 
opportunities to collaborate with 
faculty. They can provide a valuable 
service to faculty and enhance the 
scholarly value and intellectual rewards 
of their own work” (Delaney 2009, p. 35). 


Colleges and universities rely on 
research professionals for the 
collection, reporting, and analysis 

of institutional data to support 
decision-making and to provide for 
accountability. These institutional 
researchers work with data sets 

that address a wide range of topics, 
including budgets, personnel, students, 
and square footage of lab space to 
develop an overall portrait of how 
the institution works. Given the range 
of research topics, it is not surprising 
that the educational background of 
institutional researchers is anything 
but standard. Nationally, 30 percent of 
all institutional researchers hold their 
highest degree in the social sciences, 
with another 60 percent distributed 
fairly evenly across education, STEM 
fields, and business. The remaining 

10 percent are found in humanities 
and other disciplines (Volkwein, Liu, 


&Woodell, 2012). The committee had 
access through its two Institutional 
Research and Planning (IRP) members 
to the expertise of the full IRP group 

at Ohio State; members of that group 
hold advanced degrees in public affairs, 
business administration, psychology, 
library sciences, and higher education 
and student affairs. This combination of 
education yields a team that is trained 
in quantitative and qualitative research, 
program evaluation, bibliometrics, 
semantic analysis, and project 
management. With an average tenure 
at Ohio State of more than ten years, 
the institutional research (IR) staff also 
has valuable historical knowledge of 
the university. 


The faculty expertise on this committee 
was highly quantitative. Because they 
come from science and engineering 
backgrounds, the faculty members 
were able to think carefully about 
confounding variables, outliers, 
methods for pooling data, and 

specifics of statistical analysis. Their 
understanding of how their work is 
represented by institutional data and 
how that representation can be skewed 
by faculty whose work lies outside 

his or her disciplinary norms was 
tremendously important to this study. 
This evaluation required us not only 

to analyze existing data, but also to 
delve into local department culture to 
understand those data. 


IRP staff have worked closely with 
Ohio State faculty over the years, most 
commonly providing, as suggested 

by Delaney, “a valuable service’— 
data and analysis to support faculty 
scholarship or for faculty bodies to 
consider as part of faculty governance 


and decision-making. This project, 
however, had a distinctly different 
collaborative approach. Committee 
members recognized from the 
beginning that both staff and faculty 
had important contributions to make, 
with staff responsible for providing 
quality data and analyses, and faculty 
responsible for ensuring that the 

data were gathered and the analyses 
were performed with the appropriate 
background understanding and 
context. The committee learned early 
in its deliberations, however, that 
faculty were not always knowledgeable 
about research and teaching norms 
in STEM disciplines other than their 
own. The process of establishing 
context for the data thus became one 
of discovery by the whole committee, 
rather than instruction from faculty 
to staff. The IRP staff and faculty were 
peers on this project, each contributing 
her or his own methodological and 
content expertise for the good of the 
committee as a whole. 


THE EVALUATION 


As the first effort to evaluate STEM 
faculty resource allocation and work 
environment by gender for our 
institution, this project required careful 
deliberation before plunging into data 
analysis. The committee members 
were highly aware of the potential 
impact of a study that included 
gender as a variable. It therefore 
became imperative that we develop 
methodologies that would stand up to 
intense scrutiny. 


The committee met every other 


week over a three-month period 
and interacted frequently via email, 
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spending the first several weeks 
discussing its charge and what kinds 
of data would be most important. The 
first decision was to concentrate on 
resource allocation to tenure-track 
faculty, because this group demands 
the greatest share of faculty resources, 
and because the institution makes 
long-term commitments to its tenured 
faculty. Our discussions at this stage 
centered on the kinds of resources 
important for faculty as well as the 
kinds of data that were available. 
Faculty members on the committee 
brought up a suite of issues for which 
centralized data simply do not exist 
(e.g., advising loads, library resources, 
and opportunities for collaboration). 
The interplay between staff and faculty 
at this stage was crucial: as faculty 
members brought up resources they 
considered crucial to their professional 
success (library subscriptions to 
certain journals, availability of certain 
equipment, access to graduate student 
assistants), IR staff found themselves 
challenged to identify appropriate 
measures from their databases. 
Ultimately, the committee settled on 
four parameters of most interest for 
resource allocation: salary, start-up 
accounts, square footage of lab space, 
and teaching loads. Not all identified 
data were available through central 
institutional repositories, but faculty 
members of the committee were able 
to identify additional data sources. This 
supplemental information included 
data on start-up accounts offered to 
incoming faculty and data on teaching 
loads for the health sciences disciplines, 
whose teaching assignments are not 
accurately captured in the institutional 
data maintained by the Office of 
Enrollment Services. These additional 
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data were available from college 
deans’ offices. Next we describe the 
discussions that led to the final analysis 
reported in Herbers and Desai (2012). 


Literature Review 

Faculty and staff on the committee 
reviewed previous scholarship related 
to gender disparities in faculty salaries, 
teaching, and service assignments 

and research productivity. Most such 
studies base their work on survey 
data, and often use the National 
Survey of Postsecondary Faculty as a 
source. Recently, the National Research 
Council released Gender Differences 

at Critical Transitions in the Careers of 
Science, Engineering and Mathematics 
Faculty, which looked not only at salary 
and workload issues, but also at the 
allocation of laboratory space and 
start-up packages (National Research 
Council, 2010). Data for this study were 
collected through a survey of tenured 
or tenure-track faculty and department 
administrators from the 89 universities 
then categorized as Carnegie Research lI. 


Perhaps the most well-known study of 
gender disparities in science disciplines 
at a single institution is the MIT study 
“A Study on the Status of Women 
Faculty in Science at MIT” (Chisholm 

et al., 1999). A faculty committee 
collected data from the Office of the 
Dean of Science and from institutional 
researchers in the MIT Planning Office. 
In addition, that faculty committee 
conducted interviews with women 
faculty and department heads. This 
study found that female faculty 
received lower pay and fewer resources 
than their male colleagues, despite 
equal professional accomplishments. 


It became apparent to the Ohio State 
University committee that many of 
the variables used in these earlier 
studies were already collected in our 
institutional data sets, with additional 
data collected by individual colleges. 
Thus, the committee decided not to 
include a faculty survey in the research 
and analysis design. 


Faculty Salaries 

IRP provided faculty salary data, 
normed to a nine-month appointment. 
In our initial review of those data, we 
discussed determinants of faculty 
salaries that complicate the kind of 
granular analysis we intended (e.g., 
rank, time in rank, discipline, scholarly 
record). Because we were most 
interested in determining whether 

a gender gap existed for salary, 

the committee needed to employ 
techniques that would eliminate 
potentially confounding factors. The 
discussion on these factors was robust 
and lively. Committee members readily 
agreed that salaries for assistant 
professors are relatively uncomplicated, 
reflecting starting salaries at market 
and a short timespan within that rank. 
Thus evaluating salaries for assistant 
professors required that we control for 
market conditions but relatively little 
else. 


By contrast, salaries for tenured senior 
faculty reflect a multitude of factors: 

(1) salary compression, which results 
when raise pools do not keep pace with 
market increases in starting salaries; 

(2) time in rank, which should reflect 
the number of raise cycles leading to 
higher salaries for those promoted 
years ago; (3) the system of merit raises 
employed by the institution, which 


produces variation within a cohort 
that reflects differential productivity; 
(4) other market forces (e.g., external 
offers, hires from other universities, 
previous administrative experience, 
demand for a particular focus within a 
discipline), which lend further nuance 
to data interpretation. In order to focus 
on the variable of greatest interest, 
gender, we had to control for other 
factors that affect faculty salaries. 


The committee started by eliminating 
outliers (e.g., administrators including 
deans, associate deans, and chairs) as 
well as individuals holding endowed 
professorships and those with recent 
prior administrative experience 

(e.g., former deans). Even though 

the salary data were deidentified by 
IR staff, the faculty representatives 
were able to infer identities of many 
individuals involved; the discussions 
that devolved allowed us to make 
valid decisions on potential outliers. In 
particular, the faculty representatives 
were able to note individuals who had 
been hired or named as University 
Distinguished Professors or Eminent 
Scholars. To a considerable extent, 
this identification of especially stellar 
faculty as outliers reduced the effect 
of other market forces. 


The initial data analysis was granular, 
but it quickly became apparent that 
some pooling would be needed 
because of the paucity of women 

in some departments or ranks. 

The committee therefore spent 
considerable time discussing how 

to aggregate faculty into groups for 
statistical analysis. Time in rank for 
associate professors can reflect norms 
about how long individuals serve in 


that rank. Conversations among the 
faculty representatives revealed that 
the expectation for the length of time 
for promotion to professor in some 
units is within five years, whereas the 
norm in other units is within seven 

or eight years. Committee members 
agreed that faculty members who 

had been associate professors for 11 
years or longer were likely to be less 
productive than those who had been 
promoted within that time frame. Thus, 
pooling among associate professors 
should reflect variation in the norms for 
promotion to professor as well as the 
shared perception of lower productivity 
for long time at that rank. 


Similarly, faculty revealed during 

the group discussions that some 
departments routinely hire at the 
professor rank, while others do so 
rarely. While this is information that 
can be discovered and confirmed 
from the data, this hiring practice was 
not known to the IRP or the Human 
Resources staff. Furthermore, this 
information would not have been 
established without the faculty input. 
The institutional data showed time 

in rank only at Ohio State, and thus 
were only interpretable for those who 
had spent most of their career at our 
institution. One of the more difficult 
topics for salary analysis was how 

to pool groups of professors. After 
considerable discussion, the committee 
settled on the following seven 
categories of professors: 


1. Assistant professors 

2. Associate professors in rank 0-5 
years 

3. Associate professors in rank 6-11 
years 


4. Associate professors in rank 12+ 
years 

5. Professors in rank 0-5 years 

6. Professors in rank 6-11 years 

7. Professors in rank 12+ years 


The committee also spent considerable 
time discussing market forces across 
disciplines. Entry-level salaries, a 
reasonable indicator of market 
conditions, varied substantially across 
the departments the committee 
studied. The IR staff were able to 
provide an initial examination of 
those markets, and the committee 
then grouped departments that 
shared similar market forces, with the 
stipulation that departments would 
be pooled only within a college. 

For example, salaries for electrical 
engineers and computer scientists 
were comparable, as were salaries for 
geologists and ecologists. 


After these discussions, the committee 
agreed that we had a reasonably valid 
data set to examine and recommended 
evaluation of the data using multiple 
regression, with gender as both a main 
effect and an interaction term with 
rank, time in rank, and department 
group. Scholarly record was not 
included in the analysis, in part because 
the variance by rank and by disciplines 
meant that classifying faculty according 
to their level of productivity was 
problematic. 


Start-up Costs 

In the STEM disciplines newly hired 
faculty are offered funds to support 
purchase of equipment, hire laboratory 
personnel, and travel. These start-up 
accounts are negotiated as part of 

the original offer, and they can be 
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substantial: for hires at the professor 
level in the experimental sciences, 
start-up accounts exceeding $1 million 
are common. There is substantial 
variation across disciplines with regard 
to normed start-up figures, and even 
within departments there can be wide 
variation that depends on the research 
area of faculty being hired. 


Discussion revealed how difficult it 

can be to interpret those data. First, no 
standard definition exists for what is 
included in start-up costs. In addition 

to the start-up account, commitments 
to new faculty can include laboratory/ 
office renovations, major equipment 
purchases, summer salary support, 
graduate assistant support, and time 
released from teaching; all of these 
additional measures help to attract 

the best faculty and should not be 
discounted. Even so, the committee 
was forced to restrict its attention to the 
start-up account in the narrowest sense. 


The IRP staff collected start-up data 
from college offices and provided it 

in summary form to the committee. 
This was the first time the IRP staff had 
collected start-up data, and they were 
guided entirely in their collection and 
analysis by the committee faculty. 
Faculty representatives were able to 
highlight complications in the data, 
and to suggest ways to move toward 
a statistical analysis. First, the rank of 
hire mattered: senior hires demand 
and receive larger start-ups than do 
entry-level faculty. After discussion, 
the committee decided to examine 
only start-up data for new hires as 
assistant professors, which represented 
reasonable sample sizes for both men 
and women. 
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Even so, pooling of data across 
departments became important 
again. Our discussion showed that 

the categorizations of faculty by rank 
and years in rank were not necessarily 
useful for analyzing start-up accounts. 
Faculty expertise about market forces 
that determine start-up accounts 

was Critical to identify clusters of 
departments that were similar, but not 
identical, to those developed for faculty 
salaries. As an example, Chemical 

and Biomolecular; Materials Science; 
and Mechanical and Aerospace were 
clustered based on faculty advice, 
whereas Biomedical Engineering was 
kept separate. This was not a decision 
that staff could have made based on 
data available to them. 


A final additional complication derived 
from subdisciplinary differences 
within a unit; for example, theoretical 
scientists and empirical scientists 
require very different infrastructure yet 
reside within the same department. 
While acknowledging the issue, faculty 
recommended that the committee 
ignore it. 


Therefore, the final analysis of start-up 
data focused on assistant professor 
hires and became a simple 2-way 
ANOVA with gender and disciplinary 
cluster as independent variables. 


Lab Space 

Lab space is highly prized by 
experimentalists. Rooms for 
specialized equipment; bench space 
for students, postdoctoral researchers, 
and technicians; as well as ancillary 
spaces (e.g., conference rooms, 
common equipment rooms, and office 
space) contribute to faculty research 


productivity. The quality of the lab space 
matters, with clean modern lab space in 
a well-maintained building serving as an 
important recruitment tool. 


Our institution uses square footage of 
lab space as the primary datum, without 
information on quality; from the outset, 
then, the committee had to ignore 
issues such as age of the building, 

years since renovation, reliability of 
infrastructure, and other measures of 
space quality. Furthermore, centrally 
collected data on lab space assignments 
by gender were available only for faculty 
with externally funded research; new 
hires as well as more senior faculty 
without funding were not included 

in that database, and centralized data 
did not allow us to examine usage of 
shared space, equipment space, and 
office space. One of our colleges had 
conducted its own space inventory that 
was comprehensive, but for the other 
colleges the committee had incomplete 
data focusing solely on square footage 
of research laboratory space. 


The committee discussed issues 
concerning lab space assignment, 

for which decisions are local and 
idiosyncratic. A scientist with 2 
graduate students and 1 postdoctoral 
researcher requires fewer benches than 
a colleague with 15 graduate students 
and 6 postdoctoral researchers. Those 
who travel to do their research (e.g., 
tropical ecologists, field geologists, 
high-energy physicists, astronomers) 
command less space than those who 
gather data primarily in Ohio State labs. 
Thus the amount of space assigned to 
a faculty member reflects a myriad of 
variables invisible to IR staff. 


Despite those complications, the 
committee did have some a priori 
expectations. Overall, junior faculty 
require less space than senior 
colleagues, leading us to include rank 
as a covariate. Similarly, the amount of 


external funding can drive space needs. 


Finally, the ethos of space assignments 
is relatively constant within a 
department. The committee ultimately 
agreed that the data could be 
analyzed via a regression model with 
rank, gender, total external funding, 
and department as independent 
variables, but were aware of numerous 
unmeasured factors that could explain 
additional variance. 


Teaching Assignments 

Teaching includes a variety of different 
modalities, and each institution— 
and, at Ohio State, each college 

within each institution—sets its own 


definition of how teaching is measured. 


In the sciences, teaching includes 
classroom lecturing, overseeing 
graduate research assistants who offer 
laboratories and recitations, proctoring 
and grading, leading seminars and 
colloquia, supervising graduate and 
undergraduate research students, 

and supervising students in clinics. 
Disciplines vary in the distribution 

of courses offered (some have heavy 
service course responsibilities while 
others primarily teach their own 
majors), norms for team-teaching, class 
size, numbers of graduate students, 
and so on. 

The committee examined centrally 
collected data that sparked 
illuminating discussions that revealed 
widely divergent department mores. 
First, the committee learned that units 
vary substantially in how they report 


instructors of record; for example, a 
single course may be taught by four or 
five faculty, with only one recorded in 
the registrar's database. Indeed, one 

of our college representatives stressed 
that those central data failed entirely to 
capture the relevant effort information; 
for that unit the committee used 
college-supplied teaching data. 


Faculty discussions of practices in 
their own departments enlightened 
committee members, faculty, and 
staff alike as to how differently units 
handled academic-year release time. 
Faculty who secure external funding 
can use those dollars to support a 
portion of their salary, which in turn 
releases them from teaching. In some 
units, garnering such release time is 
the expectation whereas in others it is 
disallowed. Not surprisingly, faculty in 
those units for which release time is an 
expectation teach less than those in 
which it is not permitted. 


Third, considerable variation in 
teaching load derives from enrollments 
in courses collectively binned as 
independent studies courses; these 
include readings, seminars, research 
supervision, and other kinds of tutorial 
efforts. Most units hold expectations 
for faculty to engage in such activities, 
but they are rarely codified. Rather, 
faculty have broad discretion 
accepting students to their research 
groups, offering seminar courses, 

and supervising undergraduate 
internships. Furthermore, many 
faculty members supervise students 
who are not enrolled for credit. These 
kinds of teaching efforts typically are 
not assigned as part of a workload 
discussion, but rather are undertaken 


on the basis of individual faculty 
initiative. Even so, units can use data on 
independent studies teaching to make 
assignments for didactic teaching, 
such as providing a course release 

for a faculty member supervising 
independent studies for 12 graduate 
students. 


Fourth, didactic courses themselves 
vary tremendously in terms of the 
kinds of effort required of faculty. 
Large introductory courses require 
extensive administrative overhead 
(managing the course Web site, 
answering endless emails, overseeing 
teaching assistants), whereas smaller 
upper-division and graduate courses 
require a different kind of preparation. 
Committee discussions on this topic 
again uncovered variable cultures 
across disciplines in terms of the 

mix of such courses offered, as well 

as the kinds of students who enroll. 
Some units offer numerous courses 
to benefit students outside their 
discipline (general education or other 
service courses), while others serve 
primarily their own majors. After 
discussion of these complications, we 
settled on three categories of didactic 
instruction (introductory, advanced 
undergraduate, and graduate/ 
professional). 


The committee wrestled with several 
measures of teaching effort, including 
the number of courses taught, the 
number of credits hours taught, the 
number of contact hours associated 
with a course, and the number of 
students taught per term. Of those 
four, the committee settled on the first 
two as the best metrics for comparing 
teaching loads among faculty. 
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Ultimately, we settled on regression 
analysis of number of courses and 
number of credit hours as a function 
of gender, faculty rank, discipline, and 
course level. 


CONCLUSIONS 


Gender did not explain variation in our 
analyses, which is heartening (Herbers 
& Desai 2012). Most importantly, that 
central result is credible because 

of the committee discussions that 
acknowledged nuance, and decisions 
that allowed for comparisons that 
minimize confounding variables. 
Analysis of faculty workload and 
access to resources are fraught with 
difficulties (Dennison 2011), and can 
best be accomplished when IR officers 
collaborate with those whose work 
they are studying: in other words, the 
faculty. 


Our process took over a year to 
complete, including iterations of 
analyses and refinement of the baseline 
data. Committee members, each 
steeped in one discipline, continually 
learned from each other about how 
variable department cultures can be 
within one institution. Furthermore, 
we all learned about the power and 
limitations of centrally collected data 
as we strove to develop protocols 

for meaningful data analysis. Our 
experiences showed that involving 
faculty for analysis of data about their 
work is crucial to producing reliable 
and credible results. 
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